In Search of a Scalable, Parallel Branch-and-Bound for Two-Stage Stochastic Integer Optimization

Many real-world planning problems require searching for an optimal solution in the face of uncertain input. One approach to is to express them as a two-stage stochastic optimization problem where the search for an optimum in one stage is informed by the evaluation of multiple possible scenarios in the other stage. If integer solutions are required, then branchand-bound techniques are the accepted norm. However, there has been little prior work in parallelizing and scaling branch-andbound algorithms for such problems. In this paper, we explore the parallelization of a two-stage stochastic integer program solved using branch-and-bound. We present three design variations and describe the factors that shaped our quest for scalability. Our designs seek to increase the exposed parallelism while delegating sequential linear program solves to existing libraries. We evaluate the scalability of our designs using sample aircraft allocation problems for the US airfleet. These problems greatly incentivize short times to solution. Unlike typical, iterative scientific applications, we encounter some very interesting characteristics that make it challenging to realize a scalable design. The total amount of computation required to find optima is not constant across multiple runs. This challenges traditional thinking about scalability and parallel efficiency. It also implies that reducing idle time does not imply quicker runs. The sequential grains of computation are quite coarse. They display a wide variation and unpredictability in sizes. The structure of the branch-and-bound search tree is sensitive to several factors, any of which can significantly alter the search tree causing longer times to solution. We explore the causes for this fragility and evaluate the trade-offs between scalability and repeatability. Our attempts result in strong scaling to hundreds of cores for the small, sample datasets that we use. We believe our experiences will feed usefully into further research on this topic. Optimization problems are of real-world relevance, often with multi-million dollar consequences. However, real-world problems often have a level of uncertainty in inputs, environmental factors, objectives etc. that should influence the optimal solution. Stochastic optimization captures this uncertainty by formulating problems as two-stage (or multi-stage) models that propose candidate solutions and evaluate them across a spectrum of potential scenarios. In this paper we present our quest for the scalable, parallel solution of stochastic optimization problems. Specifically, we are interested in problems that require integer solutions, and hence, interested in Branch-and-Bound (BnB) approaches. BnB, a common, well-studied approach to finding optima for integer programs is notoriously hard to parallelize. Additionally, there has been little prior work in parallelizing or scaling two-stage, stochastic integer programs. Our work is set in the context of a US airfleet management problem where aircraft are allocated to missions under uncertain cargo movement demands (II). However, the parallel scaffolding and the techniques we have developed are relevant elsewhere too. We structure this paper in the form of a narrative that describes our findings and how they motivate successive modifications to our parallel designs. Once past the introductory sections (I–III), we discuss factors that influence the design of a parallel, stochastic integer program (IV). This sections makes for interesting reading and presents some of the factors that set this problem apart from typical computational science applications. We pick a programming model that enables the expression and management of the available parallelism (V). Finally, in sections VI–VIII, we present the motivations, structure and performance of each design.